Adhering substance detection apparatus and adhering substance detection method

ABSTRACT

An adhering substance detection apparatus includes calculating, first and second detecting, and generating units. The calculating unit calculates a variation in a feature value related to luminance in past and current captured images captured by an image capturing device, based on the luminance of pixels included in the captured images. The first detecting unit detects a first region in which the variation calculated by the calculating unit falls within a threshold range, and the feature value in the current captured image falls within a threshold range. The second detecting unit detects a second region in which an irregularity in a distribution of the luminance of the pixels included in the captured image satisfies a predetermined irregularity condition. The generating unit generates a sum region being a sum of the first and second regions, as an adhering substance region corresponding to an adhering substance adhering to the image capturing device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims benefit of priority of theprior Japanese Patent Application No. 2018-246916, filed on Dec. 28,2018, the entire contents of which are incorporated herein by reference.

FIELD

The embodiment discussed herein is directed to an adhering substancedetection apparatus and an adhering substance detection method.

BACKGROUND

Conventionally having been known is an adhering substance detectionapparatus that detects an adhering substance adhering to a lens, basedon a temporal change in the luminance values of segments that aredivisions of the area of the captured image (see Japanese Laid-openPatent Publication No. 2014-30188, for example).

However, in the conventional technology, there has been some room forimprovement in highly accurate detection of an adhering substance. Forexample, when the adhering substance is a mixture of a water drop andmud, the region corresponding to the water drop has higher luminancethan the region corresponding to the mud. Therefore, in of attempt todetect one of such regions, the detection of the other may fail.

SUMMARY

An adhering substance detection apparatus according to an embodimentincludes a calculating unit, a first detecting unit, a second detectingunit, and a generating unit. The calculating unit calculates a variationin a feature value related to luminance in past and current capturedimages captured by an image capturing device, based on the luminance ofpixels included in the captured images. The first detecting unit detectsa first region in which the variation calculated by the calculating unitfalls within a predetermined threshold range, and the feature value inthe current captured image fails within a predetermined threshold range.The second detecting unit detects a second region in which anirregularity in a distribution of the luminance of the pixels includedin the captured image satisfies a predetermined irregularity condition.The generating unit generates a sum region that is a sum of the firstregion detected by the first detecting unit and a second region detectedby the second detecting unit, as an adhering substance regioncorresponding to an adhering substance adhering to the image capturingdevice.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic giving a general outline of an adhering substancedetection method according to an embodiment;

FIG. 2 is a block diagram illustrating a configuration an adheringsubstance detection apparatus according to the embodiment;

FIG. 3 is a schematic illustrating the details of a process performed bya calculating unit;

FIG. 4 is a schematic illustrating a relation between a mean value ofthe luminance in a region of interest, and a first predetermined value;

FIG. 5 is a schematic for explaining pixel arrays for which luminancedistributions are extracted;

FIG. 6 is a schematic illustrating the details of a process performed bya second detecting unit;

FIG. 7 is a schematic illustrating the details of the process performedby the second detecting unit;

FIG. 8 is a schematic illustrating the details of the process performedby the second detecting unit;

FIG. 9 is a schematic illustrating the details of the process performedby the second detecting unit;

FIG. 10 is a schematic illustrating a second region conversion processperformed by the second detecting unit;

FIG. 11 is a schematic illustrating the details of a process performedby a generating unit;

FIG. 12 is a schematic illustrating the details of a process performedby a removal determining unit;

FIG. 13 is a schematic illustrating the details of the process performedby the removal determining unit;

FIG. 14 is a flowchart illustrating the sequence of an adhesiondetecting process performed by the adhering substance detectionapparatus according to the embodiment; and

FIG. 15 is a flowchart illustrating the sequence of a removaldetermination process performed by the adhering substance detectionapparatus according to the embodiment.

DESCRIPTION OF EMBODIMENT

An adhering substance detection apparatus and an adhering substancedetection method according to an embodiment of the present inventionwill now be explained in detail with reference to the appended drawings.However, the embodiment described below is not intended to limit thescope of the present invention in any way.

To begin with, a general outline of an adhering substance detectionmethod according to an embodiment will be explained with reference toFIG. 1. FIG. 1 is a schematic giving a general outline of the adheringsubstance detection method according to the embodiment. FIG. 1illustrates a captured image I that is captured by a camera that isonboard a vehicle, with some adhering substance that is a mixture ofwater and mud adhering to the lens of the camera, for example. In thecaptured image I illustrated in FIG. 1, while a region containing alarger amount of mud (mud illustrated in FIG. 1) is represented as ablack region with no gradation, because such a region penetrates almostno light, a region containing a larger amount of water (waterillustrated in FIG. 1) is represented as a blurred region, because sucha region penetrates a slight amount of light. In other words, the regioncontaining a larger amount of water has higher luminance, compared withthe region containing a larger amount of mud. Examples of the adheringsubstance are not limited to water and mud, and may include any adheringsubstance represented as a black region with no gradation, and a blurredregion in the captured image I.

Conventionally, when the adhering substance is substance such as amixture of mud and water, if the luminance threshold is set to a levelfor detecting the mud, the region blurred with water may not bedetected, because the threshold is set to a low level. If the luminancethreshold is set to a level for detecting both of the water and the mud,a larger number of objects other than the adhering substance may bedetected erroneously, because the threshold is set to a high level.Examples of the blurred region include a region including blurredobjects in the background, a region having become blurred due to thedifferent concentrations of the mud contained in the water, and a regionhaving become blurred due to the three-dimensional shape of a waterdrop, for example.

To address this issue, the adhering substance detection apparatus 1according to the embodiment (see FIG. 2) detects adhering substance suchas a mixture of water and mud highly accurately, by performing anadhering substance detection method. Specifically, the adheringsubstance detection method according to the embodiment performs a firstdetection process for detecting a black region with no gradation, whichcorresponds to mud, for example, and a second detection process fordetecting a blurred region, which corresponds to water, for example. Thesum of the two types of regions detected by these two detectionprocesses is then established as an adhering substance region. Theadhering substance detection method according to the embodiment will nowbe explained with reference to FIG. 1.

As illustrated in FIG. 1, to begin with, the adhering substancedetection method according to the embodiment calculates a variation in afeature value related to the luminance in past and current capturedimages I captured by the camera, based on the luminance of the pixelsincluded in the captured images I (S1). The feature value includes arepresentative value of the luminance and a dispersion of the luminanceof a predetermined region. Specifically, a mean value is used as therepresentative value, and a standard deviation is used as thedispersion. The variation is the difference between a feature valuecalculated from the past captured image I and a feature value calculatedfrom the current captured image I, for example.

The adhering substance detection method according to the embodiment thendetects a first region A1 in which the calculated variation in thefeature value falls within a predetermined threshold range, and in whichthe feature value in the current captured image I falls within apredetermined threshold range (S2). In other words, by performing thefirst detection process, the adhering substance detection methodaccording to the embodiment detects a first region A1 that is a blackregion with no gradation, in which the feature value has gone throughlittle variation from the past to the present, and in which the currentfeature values are small. The details of the method for detecting thefirst region A1 will be described later.

The adhering substance detection method according to the embodiment thendetects a second region A2 in which an irregularity in the distributionof the luminance of the pixels included in the current captured image Isatisfies a predetermined irregularity condition (S3). Specifically, byperforming the second detection process, the adhering substancedetection method according to the embodiment detects a region in whichthe irregularity in the luminance distribution is moderate, that is, ablurred region, as a second region A2. More specifically, the secondregion A2 can be also said to be a region in which the variation in thefeature value falls within the predetermined threshold range and thefeature value in the current captured image is outside the predeterminedthreshold range. Details of the method for detecting the second regionA2 will be described later. The distribution of the luminance of thepixels herein means a shape in which the luminance changes along apredetermined direction of a subject image. For example, defining apredetermined coordinate (x0, y0) of the image as a point of origin, anddenoting the luminance of the pixels in a horizontal direction x asL(x), the shape drawn as a graph of x-L(x) is referred to as a pixelluminance distribution in the horizontal direction, with the point oforigin at (x0, y0). x0, y0 may be set to any coordinates, and thedirection may be set to any direction at any angle, including a verticaldirection.

The adhering substance detection method according to the embodiment thengenerates a sum region that is the sum of the detected first region A1and the second region A2, as an adhering substance region A12corresponding to the adhering substance (S4).

In other words, the adhering substance detection method according to theembodiment separately detects the first region A1 that is a black regionwith no gradation, which corresponds to the mud, and detects the secondregion A2 that is a blurred region, which corresponds to the water, anddetects the adhering substance region A12 by taking the sum of the firstregion A1 and the second region A2 at the final step.

In this manner, even if the adhering substance is a mixture of mud andwater, for example, it is possible to detect the mud and the water, byperforming separate detection processes suitable for the characteristicsof the two. In other words, with the adhering substance detection methodaccording to the embodiment, adhering substance can be detected highlyaccurately.

The adhering substance detection method according to the embodiment setsa plurality of segments to the captured image I, and generates theadhering substance region A12 correspondingly to the segments, but thispoint will be described later.

Furthermore, the adhering substance detection method according to theembodiment then determines whether the adhering substance has beenremoved based only on the variation in the feature value related to theluminance in the adhering substance region A12, unlike the determinationas to whether the adhering substance region A12 adheres to. This pointwill be described later.

A configuration of the adhering substance detection apparatus 1according to the embodiment will now be explained with reference to FIG.2. FIG. 2 is a block diagram illustrating a configuration of theadhering substance detection apparatus 1 according to the embodiment. Asillustrated in FIG. 2, the adhering substance detection apparatus 1according to the embodiment is connected to a camera 10, a vehicle-speedsensor 11, and various devices 50. In the example illustrated in FIG. 2,the adhering substance detection apparatus 1 is provided separately fromthe camera 10 and the various devices 50, but the configuration is notlimited thereto, and the adhering substance detection apparatus 1 may beintegrated with at least one of the camera 10 and the various devices50.

The camera 10 is a camera that is onboard a vehicle, and is providedwith a lens such as a fisheye lens, and an imaging device such as acharge-coupled device (COD) or a complementary metal oxide-semiconductor(CMOS). The camera 10 is provided at each position where images of thefront and the rear sides, and the lateral sides of the vehicle can becaptured, for example, and the captured images I are output to theadhering substance detection apparatus 1.

The various devices 50 are devices that perform various vehicle controlby acquiring detection results of the adhering substance detectionapparatus 1. The various devices 50 include a display device fornotifying a user of the presence of an adhering substance adhering tothe lens of the camera 10 or of an instruction for wiping the adheringsubstance, a removing device for removing the adhering substance byspraying fluid, gas, or the like toward the lens, and a vehicle controldevice for controlling automated driving and the like, for example.

As illustrated in FIG. 2, the adhering substance detection apparatus 1according to the embodiment includes a control unit 2 and a storage unit3. The control unit 2 includes an acquiring unit 21, a calculating unit22, a first detecting unit 23, a second detecting unit 24, a generatingunit 25, a removal determining unit 26, and a flag output unit 27. Thestorage unit 3 stores therein irregularity condition information 31.

The adhering substance detection apparatus 1 includes a computer orvarious types of circuits including a central processing unit (CPU), aread-only memory (ROM), a random-access memory (RAM), a data flashmemory, and an input/output port.

The CPU included in the computer functions as the acquiring unit 21, thecalculating unit 22, the first detecting unit 23, the second detectingunit 24, the generating unit 25, the removal determining unit 26, andthe flag output unit 27 included in the control unit 2, by reading andexecuting a computer program stored in the ROM, for example.

At least one of or the whole of the acquiring unit 21, the calculatingunit 22, the first detecting unit 23, the second detecting unit 24, thegenerating unit 25, the removal determining unit 26, and the flag outputunit 27 included in the control unit 2 may be implemented as hardwaresuch as an application specific integrated circuit (ASIC) or afield-programmable gate array (FPGA).

The storage unit 3 corresponds to, for example, a RAM or a data flashmemory. The RAM or the data flash memory is capable of storing thereinthe irregularity condition information 31, information of variouscomputer programs, and the like. The adhering substance detectionapparatus 1 may also acquire these computer programs or various types ofinformation from another computer connected over a wired or wirelessnetwork, or via a portable recording medium.

The irregularity condition information 31 stored in the storage unit 3is information including conditions that are used as a reference in adetection process performed by the second detecting unit 24, which willbe described later, and includes a pattern condition for theirregularity in a luminance distribution, for example. A patterncondition is a pattern of the shape of the irregularity that is a map ofthe luminance distribution, or a pattern of a luminance data sequencesin the luminance distribution. The detection process using theirregularity condition information 31 will be described later.

The acquiring unit 21 acquires various types of information. Theacquiring unit 21 acquires an image captured by the camera 10, andgenerates (acquires) a current frame that is the current captured imageI. Specifically, the acquiring unit 21 performs a gray-scaling processfor converting each pixel. of the acquired image into a gray scale valuebetween white and black, based on the luminance of the pixel.

The acquiring unit 21 also performs a pixel decimation process to theacquired image, and generates an image having a smaller size than theacquired image. The acquiring unit 21 then generates a current framethat is an integral image of the sum and the sum of squares of the pixelvalues of the pixels, based on the image applied with the decimationprocess. A pixel value is information corresponding to luminance or anedge included in the pixel.

In this manner, by performing a decimation process to the acquiredimage, and generating an integral image, the adhering substancedetection apparatus can increase the calculation speed of the subsequentprocesses. Therefore, it is possible to reduce the processing time fordetecting adhering substance.

The acquiring unit 21 may also perform a smoothing process using asmoothing filter such as a mean filter. It is also possible for theacquiring unit 21 to generate a current frame having the same size asthe acquired image, without applying the decimation process.

The acquiring unit 21 also acquires a vehicle speed, based on a signalfrom the vehicle-speed sensor 11.

The calculating unit 22 calculates a variation in a feature valuerelated to the luminance in the past and current captured images Iacquired by the acquiring unit 21, based on the luminance of the pixelsincluded in the captured images I. FIG. 3 a schematic illustrating thedetails of a process performed by the calculating unit 22.

As illustrated in FIG. 3, to begin with, the calculating unit 22 sets aregion of interest ROI and segments 100 to the captured image I. Theregion of interest ROI is a rectangular region that is set in advancebased on the characteristics of the camera 10, and is a region excludingthe vehicle body region, for example. The segments 100 are rectangularregions resultant of dividing the region of interest ROI horizontallyand vertically. For example, each of the segments 100 is a regionincluding 40×40 pixels, but the number of pixels included in one segment100 may be set to any number.

The calculating unit 22 calculates a feature value related to theluminance, for each of the segments 100. Specifically, the calculatingunit 22 calculates a representative value of the luminance and adispersion of the luminance as the feature value. The representativevalue is a value indicating representative luminance in the luminancedistribution corresponding to a subject region, and, specifically, amean value is used. Without limitation to the mean value, trimmed mean,median, or a mode, for example, may also be used. A dispersion is avalue indicating the spread of the luminance distribution correspondingto a subject region, and, specifically, a standard deviation is used forthat. Without limitation to a standard deviation, a variance, amaximum/minimum width, an interquartile width, or any percentile widthmay be used. Hereinafter, an example in which a mean value is used asthe representative value, and a standard deviation is used as thedispersion will be explained. The calculating unit 22 also calculatesthese feature values related to the luminance, for the entire region ofinterest ROI.

The calculating unit 22 then calculates a variation in the feature valuein the past and current captured images I. Specifically, the calculatingunit 22 calculates a first difference that is a difference between amean value of the luminance in a segment 100 in the current capturedimage I and a mean value of the luminance in the segment 100 that is atthe same position in the past image, as a variation. In other words, thecalculating unit 22 calculates the first difference between a currentmean value and a past mean value of the luminance in the respectivesegments 100, as a variation.

The calculating unit 22 then calculates a second difference that is adifference between a standard deviation of the luminance in a segment100 included in the current captured image I, and a standard deviationof the luminance in the segment 100 that is at the same position in thepast image. In other words, the calculating unit 22 calculates thesecond difference between the standard deviation of the past luminanceand that of the current luminance in the corresponding segment 100, as avariation. Hereinafter, the past captured image I will be sometimesreferred to as a past frame I0, and the current captured image I will besometimes referred to as a current frame I1.

Referring back to FIG. 2, the first ng unit 23 will now be explained.The first detecting unit 23 detects the first region A1 in which thevariation in the feature value calculated by the calculating unit 22falls within a predetermined threshold range, and in which the featurevalue in the current captured image I falls within a predeterminedthreshold range.

Specifically, the first detecting unit 23 determines whether each of thesegments 100 is a first region A1 that satisfies a predeterminedcondition. In other words, the first detecting unit 23 detects the firstregion A1 correspondingly to the segment 100.

For example, the first detecting unit 23 determines that a segment 100satisfies the predetermined condition if the conditions (1) to (3)described below are all satisfied, and detects the segment 100 as thefirst region A1. If at least one of the conditions (1) to (3) is notsatisfied, the first detecting unit 23 determines that the segment 100does not satisfy the predetermined condition, and detects the segment100 as a region to which no adhering substance adheres.

(1) The first difference is equal to or less than a first predetermineddifference;

(2) the second difference is equal to or less than a secondpredetermined difference; and

(3) the mean value: of the luminance in the segment 100 is equal to orless than a first predetermined value.

The condition (1) is a condition for determining whether the degree ofchange in the luminance is small in the same segment 100. The firstpredetermined difference in the condition (1) sets an upper bound to thedifference in the mean values of the luminance when the adheringsubstance adheres to, and is a difference that is set in advance basedon experiments or the like. The first predetermined difference is “5”,for example.

The condition (2) is a condition for suppressing the impact of the gainadjustment of the camera 10. The second predetermined difference in thecondition (2) sets an upper bound to the difference in the luminancestandard deviation when the adhering substance adheres to, and is adifference that is set in advance based on experiments or the like. Thesecond predetermined difference is “1”, for example.

The condition (3) is a condition for determining whether the luminanceof the segment 100 in the current frame I1 is at a low level. The firstpredetermined value in the condition (3) is a value for allowing thepresence of the adhering substance in the segment 100 to be determined,and is a value that is set based on the mean value of the luminance inthe region of interest ROI included in the current frame I1, asillustrated in FIG. 4.

FIG. 4 is a schematic illustrating a relation between the mean value ofthe luminance in the region of interest ROI and the first predeterminedvalue. When the mean value of the luminance in he region of interest ROIis higher, the first predetermined value is set higher. An upper boundis set to the first predetermined value, and the first predeterminedvalue is set to the upper bound when the mean value of the luminance inthe region of interest ROI is equal to or higher than a predeterminedmean value that is set in advance. In this manner, even if the meanluminance value changes due to the gain adjustment carried out by thecamera 10 based on the background of the captured image I, the firstregion A1 can be detected highly accurately.

By detecting a segment 100 that satisfies conditions (1) to (3) as afirst region A1, the first detecting unit 23 can determine a segment 100in which the adhering substance is present, correctly. Therefore, it ispossible to improve the accuracy of the adhering substance region A12that is generated at the final step.

The first detecting unit 23 may also calculate a counter valueindicating the continuity of the satisfactions of the conditions (1) to(3), for each of the segments 100, and detect the segment 100 as thefirst region A1 when the counter value becomes equal to or greater thana predetermined threshold. Specifically, the first detecting unit 23increments the current counter value if it is determined that thesegment 100 in the current frame I1 satisfies the conditions (1) to (3),and decrements the current counter value if it is determined that thesegment 100 in the current frame I1 does not satisfy the conditions (1)to (3). In other words, the first. detecting unit 23 updates the countervalue in the segment 100.

The counter value has an upper-bound counter value and a lower-boundcounter value that are set in advance. The value by which the countervalue is incremented and decremented every time the determination ismade may be the same or different.

Furthermore, the first detecting unit 23 may also perform the process ofdetecting the first region A1 only when the speed of the vehiclesatisfies a predetermined condition. For example, the first detectingunit 23 may be caused to perform the detection process if the speed ofthe vehicle is equal to or lower than a predetermined vehicle speed. Thepredetermined vehicle speed is a vehicle speed that is set in advance,and is a vehicle speed at which the camera 10 is able to capture acaptured image I from which the adhering substance is detectable, with asmall amount of blur in the captured image I. For example, thepredetermined vehicle speed is 80 km/h. In this manner, she first regionA1 can be detected highly accurately.

Alternatively, the first detecting unit 23 may be configured to performthe detection process if the vehicle C is moving, specifically, if thevehicle speed is equal to or higher than a low vehicle speed that is setin advance. In this manner, it is possible to prevent the detectionprocess to be performed repeatedly, when the vehicle C has stopped andthe same captured image I has been captured repeatedly.

Furthermore, the first detecting unit 23 may be configured not toperform the process of detecting the first region A1 if the currentframe I1 is a low-illuminance image, so that the generating unit 25 isnot caused to generate the adhering substance region A12 at thesubsequent stage. A low-illuminance image is a captured image I that iscaptured while the environment around the vehicle C is dark, e.g., whilethe vehicle C is driving during the nighttime or inside a tunnel.

The first detecting unit 23 determines that the current frame I1 is alow-illuminance image when the mean value of the luminance in the regionof interest ROI included in the current frame I1 is equal to or lowerthan a predetermined low illuminance value, and the standard deviationof the luminance in the region of interest ROI included in the currentframe I1 is equal to or less than a predetermined low illuminancedeviation. The predetermined low illuminance value is a value that isset in advance, and is “85”, for example. The predetermined lowilluminance deviation is also a value that is set in advance, and is“50”, for example.

In this manner, it is possible to suppress misdetection of an adheringsubstance adhering to the lens of the camera 10, when the image iscaptured in a low illuminance environment. Furthermore, by not causingthe adhering substance detection apparatus 1 to perform the detectionprocess when the current frame I1 is a low-illuminance image in whichthe adhering substance region A12 may not be detected correctly, theprocessing load can be suppressed.

Referring back to FIG. 2, the second detecting unit 24 will now beexplained. The second detecting unit 24 detects a second region A2 inwhich the irregularity in the pixel luminance distribution in thecaptured image I satisfies a predetermined irregularity condition.Specifically, to begin with, the second detecting unit 24 extracts acandidate region 200 that is a candidate of the second region A2, fromthe captured image I acquired by the acquiring unit 21. Specifically, tobegin with, the second detecting unit 24 extracts luminance informationof the pixels, and edge information from the captured image I. Theluminance of each pixel is expressed as a parameter ranging from 0 to255, for example.

The second detecting unit 24 also detect an edge in the X-axis direction(the right-and-left direction of the captured image I) and an edge inthe Y-axis direction (up-and-down direction of the captured image I)from each of the pixels, by performing an edge detection process basedon the luminance of the pixels. In the edge detection process, any edgedetection filter such as a Sobel filter or a Prewitt filter may be used.

The second detecting unit 24 then detects, as the edge information, avector including an edge angle and edge strength information of thepixel, using a trigonometric function based on the edge in the X-axisdirection and the edge in the Y-axis direction. Specifically, an edgeangle is expressed as an orientation of the vector, and an edge strengthis expressed as a length of the vector.

The second detecting unit 24 then performs a matching process (templatematching) for matching the detected edge information with templateinformation representing the contours of adhering substance, andprepared in advance, and extracts edge information that is similar tothe template information. The second detecting unit 24 then extracts theregion corresponding to the extracted edge information, that is, thecandidate region 200 that is a rectangular region. including the contourof a blurred region that is a second region A2.

Because the candidate region 200 is a rectangular region surrounding theregion including the matching edge information, unlike the segment 100described above, the candidate region 200 have various sizes dependingon the matching result. Furthermore, a plurality of candidate regions200 may overlap each other.

The second detecting unit 24 then extracts the luminance distributioncorresponding to a predetermined pixel array that is included in theextracted candidate region 200. FIG. 5 is a schematic for explainingpixel arrays for which luminance distributions are extracted. Asillustrated in FIG. 5, for each of the candidate regions 200 extractedfrom the captured image I, the second detecting unit 24 extracts theluminance distributions corresponding to three pixel arrays H1 to H3 inthe horizontal direction, and of three pixel arrays V1 to V3 in thevertical direction.

The extracted pixel arrays may be the pixel arrays in at least one ofthe horizontal and the vertical directions. Furthermore, the number ofpixel arrays to be extracted may be two or less, or four or more,without limitation to three.

The second detecting unit 24 then divides the extracted candidate region200 into a predetermined number of unit regions, and calculates arepresentative value of the luminance, for each of the unit regions. Themethod by which the second detecting unit 24 calculates therepresentative value will be described later with reference to FIGS. 6and 7.

The second detecting unit 24 converts the luminance of each pixelincluded in the candidate region 200 into a luminance unit representinga predetermined luminance range as a unit. For example, the seconddetecting unit 24 converts a parameter representing luminance within therange of 0 to 255 into a luminance unit that is a division of thisparameter range at a predetermined interval.

FIGS. 6 and 7 are schematics illustrating the details of the processperformed by the second detecting unit 24. To begin with, a method bywhich the second detecting unit 24 sets the unit regions will beexplained with reference to FIG. 6. FIG. 6 illustrates the distributionof the luminance in one of the horizontal pixel arrays H.

As illustrated in FIG. 6, the second detecting unit 24 divides thehorizontal pixel array into eight unit regions R1 to R8 (sometimescollectively referred to as unit regions R), for example. Each of theunit regions R1 to R8 may have the same width (the same number ofpixels) (that is, the number of pixels that is an equal division of thepixel array), or have different widths from those the others.

The number of unit regions R into which the pixel array is divided isnot limited to eight, and may be set to any number. It is preferable tokeep the number of the unit regions R into which the pixel array isdivided constant (eight, in FIG. 4), regardless of the size of thecandidate regions 200 extracted from the captured image I. In thismanner, although the extracted candidate regions 200 have various sizes,unified information can be obtained by keeping the number of unitregions R constant. Therefore, it is possible to suppress the processingload in the subsequent processes such as a determination process.

The second detecting unit 24 then calculates a representative value ofthe luminance in each of the unit regions R, as illustrated in FIG. 7.As illustrated in the top graph in FIG. 7, the second detecting unit 24converts the luminance value of each pixel (e.g., ranging from 0 to 255)into a luminance unit, before calculating the representative value.Specifically, in FIG. 7, the range 0 to 255 is divided into eightluminance units. The luminance units are denoted as “0” to “7”,respectively, the middle graph in FIG. 7. In this example, the width ofthe luminance values is divided at an interval of 32. For example, theluminance unit “0” corresponds to the luminance values 0 to 31, and theluminance unit “1” corresponds to the luminance values 32 to 63. Inother words, this conversion into a luminance unit is a process ofreducing the luminance resolution. In this manner, the resolution of theluminance distribution can be reduced to the number of desirableluminance units. Therefore, it is possible to reduce the processing loadin the subsequent processes. In the conversion to a luminance unit, thenumber of luminance units into which the luminance range is divided, andthe width of the luminance unit may be set any way. Furthermore, it isnot necessary for the luminance units to have equal widths.

The second detecting unit 24 then creates a histogram of luminanceunits, for each of the unit regions R1 to R8. The middle graph in FIG. 7illustrates the histogram for the unit region R1. In this histogram, theluminance units “0” to “7” are assigned as the classes, and the numberof pixels are assigned as the frequency.

The second detecting unit 24 then calculates, for each of the unitregions R1 to R8, a representative luminance value based on the createdhistogram, as illustrated in the bottom graph in FIG. 7. For example,the second detecting unit 24 calculates the luminance unit correspondingto the class with the highest frequency in the histogram (class “3” inFIG. 7) as the representative luminance value in the unit region R1. Inthis manner, because the number of data pieces in the luminancedistribution can be reduced, from the number of pixels to the numberunit regions R, the processing load in the subsequent processes can bereduced.

The second detecting unit 24 has been explained to calculate theluminance unit appearing at the highest frequency as the representativevalue, but without limitation thereto, the median, the mean value, andthe like in the histogram may also be used as the representative value.

Furthermore, without limitation to the calculation of the representativevalue based on the histogram, the second detecting unit 24 may alsocalculate a mean value from the luminance values, for each of the unitregions R, and use the luminance unit corresponding to the mean value asthe representative luminance value, for example.

Furthermore, the second detecting unit 24 has been explained to use aluminance unit as the representative value, but may also use the meanvalue or the like of the luminance values in the unit region R, as therepresentative value, as it is. In other words, the representative valuemay be expressed as a luminance unit or as a luminance value.

The second detecting unit 24 then determines whether the candidateregion 200 is a second region A2 based on the irregularity in the pixelluminance distribution in the candidate region 200. The determinationprocess performed by the second detecting unit 24 will now be explainedwith reference to FIGS. 8 and 9.

FIGS. 8 and 9 are schematics illustrating the details of the processperformed by the second detecting unit 24. The top graph in FIG. 8represents a luminance distribution in the candidate region 200, and therepresentative values corresponding to the respective unit regions R1 toR8 are indicated on the white background inside of the respective bars.

To begin with, as illustrated in the upper part of FIG. 8, the seconddetecting unit 24 calculates amounts of change D1 to D7 between theadjacent unit regions R1 to R8 in the luminance units. A tablecontaining the amounts of change D1 to D7 is indicated in the lower partof FIG. 8 (upper table).

If the pattern of change followed by the irregularity in the luminancedistribution satisfies a predetermined pattern of change, the seconddetecting unit 24 determines that the candidate region 200 is a secondregion A2. Specifically, the second detecting unit 24 performs thisdetermination process by comparing each of the amounts of change D1 toD7 with the irregularity condition information 31 stored in the storageunit 3.

As an example of the irregularity condition information 31, an exampleof a table containing threshold ranges for the respective amounts ofchange D1 to D7 is indicated in the lower part of FIG. 8 (lower table).If each of the amounts of change D1 to D7 in the candidate region 200falls within the corresponding threshold range specified for the amountof change D1 to D7 in the irregularity condition information 31, thesecond detecting unit 24 determines that the candidate region 200 is asecond region A2.

In other words, if the amounts of change D1 to D7 between the adjacentunit regions R1 to R8 in the luminance units satisfy the pattern ofchange specified as the threshold ranges in the irregularity conditioninformation 31, the second detecting unit 24 determines that thecandidate region 200 is the second region A2.

In other words, before the second detecting unit 24 performs thedetermination process, the feature of blurred regions, the feature beingsuch the luminance gradually becomes higher (or lower) toward the centerof the candidate region 200, is stored as a threshold range in theirregularity condition information 31. In this manner, the seconddetecting unit 24 can detect a blurred region caused by the adhesion ofwater, as a second region A2.

Furthermore, with the use of the amounts of change D1 to D7, the seconddetecting unit 24 can ignore the difference in the scales of theluminance values. Therefore, it is possible to reduce the number oferroneous determinations made when the shapes of the irregularities aresimilar, but the luminance values are different in scales. Furthermore,because the scales of the luminance values can be ignored, it is notnecessary to establish a determination condition for each of theluminance values. Therefore, the storage capacity for storing theconditions can be reduced. Furthermore, because it is not necessary tomake the determination for each of the luminance values, the processingburden can be reduced.

Furthermore, by specifying the amounts of change D1 to D7 with somewidths by setting the maximum and the minimum thereto in theirregularity condition information 31, even if the adhering substancehas a distorted shape, such a region can be detected as an adheringsubstance region. In other words, even when the adhering substance hasdifferent shapes, such regions can be detected as adhering substanceregions highly accurately.

Illustrated in FIG. 8 is an example in which the threshold range is setto each of the amounts of change D1 to D7 in the irregularity conditioninformation 31. However, when detected are small-sized second regionsA2, it is possible to set the threshold range only for some of theamounts of change D1 to D7.

Furthermore, explained in FIG. 8 is an example in which the seconddetecting unit 24 determines whether the amounts of change fall withinthe respective threshold ranges specified in the irregularity conditioninformation 31, but the second detecting unit 24 may also perform thedetermination process based on the irregularity condition information 31in which the irregularity in the luminance distribution is mapped basedon the threshold ranges corresponding to the amounts of change D1 to D7,for example. This point will be now explained with reference to FIG. 9.

The table in the upper part of FIG. 9 indicates the threshold rangescorresponding to the amounts of change in the irregularity in theluminance distribution. The diagram in the lower part of FIG. 9illustrates the irregularity condition information 31 that is mapping ofthe threshold ranges corresponding to the amounts of change D1 to D4illustrated in the upper part of FIG. 9. Specifically, the diagram inthe lower part of FIG. 9 provides a map in which the horizontal axisrepresents the positions of the unit regions R1 to R8, and the verticalaxis represents the relative luminance. Such a map is generated inadvance.

For example, the amount of change Di is specified as a threshold rangeof +1 to +2, so two squares at predetermined positions in the relativeluminance are set as the threshold for the unit region R1. For the unitregion R2, one square at a position satisfying the threshold range ofthe amount of change D1 is set as the threshold. The amount of change D2is specified with a value +1, so a square at the level immediately abovethe square set for the unit region R2 is set as the threshold for theunit region R3. The amount of change D3 has a value of −1, so the squareat the level immediately below the square set for the unit region R3 isset as the threshold for the unit region R4. The amount of change D4 isspecified with a threshold range from −2 to −1, so the two squares atthe level immediately below the square set for the unit region R4 areset as the threshold for the unit region R5. By following these steps,the mapping of the irregularity condition information 31 is completed.

In other words, the map specified in the irregularity conditioninformation 31 is information representing the shape of the irregularityin the luminance units in the unit regions R1 to R5, mapped based on theamounts of change D1 to D4. Because no threshold range is set for theamounts of change D5 to D7, any luminance to be detected is acceptablefor the unit regions R6 to R8.

The second detecting unit 24 creates a map, based on the amounts ofchange D1 to D7 in the unit regions R1 to R8 included in the extractedcandidate region 200, following the same steps as those described above,performs a matching process of matching the map with the map of theirregularity condition information 31, and determines that the candidateregion 200 is a second region A2 if the maps match.

In the example illustrated in FIG. 9, if the map based on the candidateregion 200 has an inverted V-shape as that of the map of theirregularity condition information 31, the second detecting unit 24determines that the candidate region 200 is a second region A2 that is ablurred region in which the luminance becomes lower from the centertoward the periphery. By contrast, if the map based on the candidateregion 200 has a V-shape as that of the map of the irregularitycondition information 31, the second detecting unit 24 determines thatthe candidate region 200 is a second region A2 that is a blurred regionin which the luminance becomes higher from the center toward theperiphery.

In other words, if the irregularity in the luminance distribution in thecandidate region 200 has an inverted V-shape or a V shape, the seconddetecting unit 24 determines that the candidate region 200 is a secondregion A2. In this manner, because the determination process can beperformed depending only on the shape of the irregularity, with thefactor of the luminance values (luminance units) removed, misseddetection due to the scales of the luminance values can be reduced.Therefore, an adhering substance can be detected highly accurately.

If the second detecting unit 24 keeps determining that the candidateregion 200 is a second region A2 continuously, based on the capturedimages I captured in the temporal order, the second detecting unit 24may determine that candidate region 200 is a region ascertained as asecond region A2.

Specifically, every time the second detecting unit 24 performs adetermination process as to whether a candidate region 200 is a secondregion A2, the second detecting unit 24 assigns a score corresponding tothe determination result to the candidate region 200, for each of aplurality of candidate regions 200, and determines the candidate region200 having a total score satisfying a predetermined threshold conditionas a region ascertained as a second region A2.

More specifically, if the second detecting unit 24 determines that acandidate region 200 is a second region A2, the second detecting unit 24adds a predetermined value to the total score. If the second detectingunit 24 determines that the candidate region 200 is not a second regionA2, the second detecting unit 24 subtracts a predetermined value fromthe total score. The same predetermined value may be used for both ofthe addition and the subtraction, or different predetermined values maybe used for the addition and the subtraction.

The second detecting unit 24 then performs a conversion process ofconverting the detected second region A2 into a region having a sizecorresponding to the segments 100. This point will now be explained withreference to FIG. 10.

FIG. 10 is a schematic illustrating the conversion process of the secondregion A2, performed by the second detecting unit 24. The upper part ofFIG. 10 illustrates the second regions A2 detected from the region ofinterest ROI included in the captured image I. The lower part of FIG. 10illustrates the segments 100 set to the region of interest ROI. Thenumber and the layout of the segments 100 into which the second regionA2 is to be converted by the second detecting unit 24 are matched withthose of the segments 100 set by the first detecting unit 23 (see FIG.3).

As illustrated in FIG. 10, the second detecting unit 24 superimposes thedetected second region A2 (hereinafter, sometimes referred to as anoriginal second region A2) over the segments 100, and generates segments100 overlapping with the original second region A2 as a converted secondregion A2 (sometimes referred to as a new second region A2). In thegeneration of a new second region A2, it is preferable to consider notonly the condition that the segments 100 corresponding thereto areblurred regions, but also the condition that there has been no variationin the feature value in the segments 100. More specifically, it ispreferable to generate a segment 100 that satisfies the condition (1)mentioned above, among the conditions (1) to (3) used in thecalculations of the regions A1, and that overlaps with the originalsecond region A2, as a new second region A2. The condition (1) mentionedabove stipulates that the luminance at the segment 100 has gone throughlittle variation. In this manner, an adhering substance, such as amixture of a water drop and mud, can be detected effectively, even ifsuch a region is blurred, has gone through little luminance variation,but is not represented as a black spot without any gradation.

Specifically, if a segment 100 is occupied by the original second regionA2 by a ratio equal to or higher than a predetermined threshold, thesecond detecting unit 24 generates the segment 100 as a new secondregion A2. In other words, the second detecting unit 24 detects thesecond region A2 correspondingly to the segments 100.

Referring back to FIG. 2, the generating unit 25 will now be explained.The generating unit 25 generates a sum region that is the sum of thefirst regions A1 detected by the first detecting unit 23, and the secondregions A2 detected by the second detecting unit 24, as an adheringsubstance region A12. The details of this process performed by thegenerating unit 25 will now be explained with reference to FIG. 11.

FIG. 11 is a schematic illustrating the details of the process performedby the generating unit 25. As illustrated in FIG. 11, the generatingunit 25 generates a region that is a logical sum of the first region A1corresponding to the segments 100 and the second region A2 correspondingto the segments 100, as the adhering substance region A12. In otherwords, the generating unit 25 generates an adhering substance region A12correspondingly to the segments 100. Without simply taking a logicalsum, the condition (1) may be taken into consideration, as mentionedearlier, at this point in time. Specifically, the generating unit 25 maygenerate a new second region A2 merely by taking over the originalsecond regions A2 as they are, and exclude the segments 100 notsatisfying the condition (1) in acquiring the sum for the new secondregion A2.

In this manner, by using the adhering substance region A12, the firstregion A1, and the second region A2 all of which correspond to thesegments 100, the adhering substance region A12 can be generated usingthe unified information. Therefore, complication of the process can beavoided. Furthermore, by setting a plurality of pixels as a segment 100,the units in which the processes are performed are changed from thepixels to the segments 100. In this manner, the number of times theprocess is performed can be reduced, so that the processing load can bereduced.

The generating unit 25 then calculates an occupied ratio that is a ratioof the region of interest ROI occupied by the adhering substance regionA12. If the occupied ratio is equal to or higher than a predeterminedthreshold (e.g., 40%), the generating unit 25 generates an adheringsubstance flag ON, and outputs the signal to the flag output unit 27.

If the occupied ratio is equal to or higher than the predeterminedthreshold (e.g., 40%), the generating unit 25 also calculates thefeature value related to the luminance in the adhering substance regionA12, and outputs the feature value to the removal determining unit 26

Referring back to FIG. 2, the removal determining unit 26 will now beexplained. The removal determining unit 26 determines whether theadhering substance has been removed, based on the variation in thefeature value related to the luminance in the adhering substance regionA12 generated by the generating unit 25. The details of this processperformed by the removal determining unit 26 will now be explainedspecifically, with reference to FIGS. 12 and 13.

FIGS. 12 and 13 are schematics illustrating the details of the processperformed by the removal determining unit 26. In FIG. 12, it is assumedthat the occupied ratio occupied by the adhering substance region A12has reached a level equal to or higher than the predetermined thresholdat time t1.

The removal determining unit 26 calculates a variation between a featurevalue at the time at which the adhering substance region A12 isgenerated, and a feature value that is based on the current capturedimage I, and determines that the adhering substance has been removedwhen the variation continues to remain at a level equal to or higherthan the predetermined threshold.

Specifically, every time a new captured image I is received, the removaldetermining unit 26 calculates the feature value related to theluminance of a determination region A120 included in the new capturedimage I, and calculates a difference between the feature value in thedetermination region A120 and that in the adhering substance region A12.In the example illustrated in FIG. 12, the removal determining unit 26calculates the difference between the feature value in the adheringsubstance region A12 at the time t1, and the feature value in thedetermination region A120 at time t2, as the variation, and calculatesthe difference between the feature value in the adhering substanceregion A12 at the time t1, and the feature value in the determinationregion A120 at time t3, as the variation.

In other words, the removal determining unit 26 determines that theadhering substance has been removed only based on the variation in thefeature value in the adhering substance region A12 (the determinationregion A120), not through the detection processes performed by the firstdetecting unit 23 and the second detecting unit 24.

In this manner, because the accuracy of the removal determination doesnot depend on the detection results of both of the first detecting unit23 and the second detecting unit 24, determination errors in the removaldeterminations can be reduced.

The removal determining unit 26 may determine that the adheringsubstance has been removed if the number of times the condition“variation≥threshold” is satisfied has become equal to or more than apredetermined number of times, or may calculate a score for eachdetermination result “variation≥threshold”, and determine whether theadhering substance has been removed based on the score. This point willnow be explained, with reference to FIG. 13.

in FIG. 13, the vertical axis represents the score indicating thecontinuity related to the removal determinations, and the horizontalaxis represents time. It is assumed herein that the time t1 is the timeat which the occupied ratio occupied by the adhering substance regionA12 has become equal to or higher than the predetermined threshold. Thearrow in the solid line indicates a determination result of“variation≥threshold”, and the arrow in the dotted line indicates adetermination result of “variation<threshold”.

As illustrated in FIG. 13, the removal determining unit 26 assigns aninitial value to the score of the adhering substance region A12 at thetime t1, subtracts the score when it is determined that“variation≥threshold”, and maintains the score when it is determinedthat “variation<threshold”.

If the score drops to a level lower than a predetermined removalthreshold within a predetermined time period D between the time t1 andtime tn, the removal determining unit 26 determines that the adheringsubstance corresponding to the adhering substance region A12 has beenremoved. The predetermined time period D is a time period that is set inadvance, and is a time period allowing a determination to be made as towhether a removing operation has been performed. In this manner, if thecondition “variation≥threshold” remains being satisfied over thepredetermined time period D, it can be determined that the adheringsubstance has been removed. Therefore, it is possible to avoid making anerroneous determination when “variation≥threshold” is satisfiedtemporarily due to the noise in the captured image I, for example. Inother words, the removal determination can be performed highlyaccurately.

When the predetermined time period D expires while the score is at alevel equal to or higher than the removal threshold, the removaldetermining unit 26 sets the score to the initial value again.

When it is determined that the adhering substance corresponding to theadhering substance region A12 has been removed, the removal determiningunit 26 generate an adhering substance flag OFF, and outputs the signalto the flag output unit 27.

Referring back to FIG. 2, the flag output unit 27 will now be explained.Upon receiving an adhering substance flag ON from the generating unit25, the flag output unit 27 outputs the adhering substance flag ONindicating that the adhering substance adheres to, to the variousdevices 50. Upon receiving an adhering substance flag OFF from theremoval determining unit 26, the flag output unit 27 outputs theadhering substance flag OFF indicating that no adhering substanceadheres to, to the various devices 50.

in other words, the information indicating whether the adheringsubstance flag is ON or OFF serves as information indicating validity ofwhether the various devices 50 can use the captured image Icorresponding to the current frame, or as information indicating thereliability of control performed by the various devices 50 using thecaptured image I. Therefore, instead of the information of the adheringsubstance flag, the second detecting unit 24 may also output informationindicating the validity or the reliability of the captured image I tothe various devices 50.

The sequence of the adhesion detecting process performed by the adheringsubstance detection apparatus 1 according to the embodiment will now beexplained with reference to FIG. 14. FIG. 14 is a flowchart illustratingthe sequence of the adhesion detecting process performed by the adheringsubstance detection apparatus 1 according to the embodiment.

As illustrated in FIG. 14, to begin with, the acquiring unit 21 acquiresan image captured by the camera 10, applies a gray-scaling process and adecimation process to the acquired image, and acquires an integral imagegenerated based on the pixel values of the decimated image, as acaptured image I (S101).

The calculating unit 22 calculates a variation in the feature valuerelated to the luminance of the current and the past captured images I,based on the luminance of the pixels included in the captured images I(S102).

The first detecting unit 23 then detects a first region A1 in which thevariation calculated by the calculating unit 22 falls within apredetermined threshold range, and in which the feature value in thecurrent captured image I falls within a predetermined threshold range(S103).

The second detecting unit 24 then detects a second region A2 in whichthe irregularity in the pixel luminance distribution in the capturedimage I satisfies a predetermined irregularity condition (S104). Thegenerating unit 25 then generates a sum region that is the sum of thefirst region A1 detected by the first detecting unit 23 and the secondregion A2 detected by the second detecting unit 24, as an adheringsubstance region A12 (S105).

The flag output unit 27 then outputs the adhering substance flag ONinput from the generating unit 25 to the various devices 50 (S106), andthe process is ended.

The sequence of the removal determination. process performed by theadhering substance detection apparatus 1 according to the embodimentwill now be explained with reference to FIG. 15. FIG. 15 is a flowchartillustrating the sequence of the removal determination process performedby the adhering substance detection apparatus 1 according to theembodiment.

As illustrated in FIG. 15, the generating unit 25 calculates the featurevalue of the adhering substance region A12 (S201). The acquiring unit 21then acquires the captured image I (S202).

The removal determining unit 26 calculates the feature value in thedetermination region A120, which corresponds to the adhering substanceregion A12 (S203). The removal determining unit 26 determines whetherthe variation that is the difference between the feature value of theadhering substance region A12 and the feature value of the determinationregion A120 is equal to or higher than a predetermined threshold (S204).

If the variation is equal to or greater than a predetermined threshold(Yes at S204), the removal determining unit 26 subtracts a predeterminedvalue from the initial score (S205). The removal determining unit 26then determines whether the score is less than the removal threshold(S206).

If the score is less than the removal threshold (Yes at S206), theremoval determining unit 26 determines that the adhering substancecorresponding to the adhering substance region A12 has been removed(S207). The flag output unit 27 then outputs the adhering substance flagOFF input from the removal determining unit 26 to the various devices 50(S208), and the process is ended.

If the removal determining unit 26 determines that the variation lessthan the predetermined threshold at Step S204 (No at S204), the flagoutput unit 27 outputs the adhering substance flag ON (S209), and theprocess is ended.

If the score is equal to or higher than the removal threshold at StepS206 (No at S206), the removal determining unit 26 performs Step S209,and the process is ended.

As described above, the adhering substance detection apparatus 1according to the embodiment includes the calculating unit 22, the firstdetecting unit 23, the second detecting unit 24, and the generating unit25. The calculating unit 22 calculates a variation in the feature valuerelated to the luminance in the past and current captured images Icaptured by the camera 10, based on the luminance of the pixels includedin the captured images I. The first detecting unit 23 detects a firstregion A1 in which the variation calculated by the calculating unit 22falls within a predetermined threshold range and in which the featurevalue in the current captured image I falls within a predeterminedthreshold range. The second detecting unit 24 detects a second region A2in which the irregularity in the luminance distribution of the pixelsincluded in the captured image I satisfies a predetermined irregularitycondition. The generating unit 25 generates a sum region that is the sumof the first region A1 detected by the first detecting unit 23 and thesecond region A2 detected by the second detecting unit 24, as anadhering substance region A12 corresponding to the adhering substanceadhering to the camera 10. In this manner, adhering substance can bedetected highly accurately.

Furthermore, explained in the embodiment above is an example in whichthe captured images I captured with a camera provided on board a vehicleis used, but the captured images I may be those captured by asurveillance camera or a camera installed on a street light, forexample. In other words, the captured images I may be any capturedimages I that are captured with a camera on which some adheringsubstance can adhere to the lens of the camera.

According to the present invention, an adhering substance can bedetected highly accurately.

Although the invention has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. An adhering substance detection apparatuscomprising: a calculating unit that calculates a variation in a featurevalue related to luminance in past and current captured images capturedby an image capturing device, based on the luminance of pixels includedin the captured images; a first detecting unit that detects a firstregion in which the variation calculated by the calculating unit fallswithin a predetermined threshold range, and in which the feature valuein the current captured image falls within a predetermined thresholdrange; a second detecting unit that detects a second region in which anirregularity in a distribution of the luminance of the pixels includedin the captured image satisfies a predetermined irregularity condition;and a generating unit that generates a sum region. that is a sum of thefirst region detected by the first detecting unit, and the second regiondetected by the second detecting unit, as an adhering substance regioncorresponding to an adhering substance adhering to the image capturingdevice.
 2. The adhering substance detection apparatus according to claim1, wherein the calculating unit calculates the variation for each of aplurality of segments that are set to the captured image, the firstdetecting unit detects the first region corresponding to the segments,the second detecting unit detects the second region corresponding to thesegments, and the generating unit generates the adhering substanceregion corresponding to the segments.
 3. The adhering substancedetection apparatus according to claim 1, further comprising a removaldetermining unit that determines whether the adhering substance has beenremoved, based on the variation in the feature value related to theluminance in the adhering substance region generated by the generatingunit.
 4. The adhering substance detection apparatus according to claim3, wherein the removal determining unit calculates a variation betweenthe feature value of time at which the adhering substance region isgenerated and the feature value that is based on the current capturedimage, and determines that the adhering substance has been removed whenthe variation remains equal to or greater than a predeterminedthreshold, continuously.
 5. The adhering substance detection apparatusaccording to claim 1, wherein the calculating unit calculates arepresentative value of the luminance and a dispersion of the luminancein a predetermined region as the feature value, and calculate a firstdifference between a past representative value and a currentrepresentative value of the luminance, and a second difference between acurrent dispersion and a past dispersion of the luminance, as thevariation, and the first detecting unit detects the first region inwhich the first difference is equal to or less than a firstpredetermined difference, in which the second difference is equal to orless than a second predetermined difference, and in which the currentrepresentative value of the luminance is equal to or less than a firstpredetermined value.
 6. The adhering substance detection apparatusaccording to claim 1, wherein the second detecting unit detects thesecond region in which a pattern of change in the irregularity in thedistribution of the luminance satisfies a predetermined pattern ofchange.
 7. An adhering substance detection method comprising:calculating a variation in a feature value related to luminance in pastand current captured images captured by an image capturing device, basedon the luminance of pixels included in the captured images; detecting afirst region in which the variation calculated at the calculating fallswithin a predetermined threshold range, and in which the feature valuein the current captured image falls within a predetermined thresholdrange; detecting a second region in which an irregularity in adistribution of the luminance of the pixels included in the capturedimage satisfies a predetermined irregularity condition; generating a sumregion that a sum of the first region detected at the detecting of thefirst region, and the second region detected at the detecting of thesecond region, as an adhering substance region corresponding to anadhering substance adhering to the image capturing device.